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appropriate responses in that context; knowledge systeis was used to 
mean sets of related facts that are commonly taught as a body of 
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ABSTRACT 



Academic research on computer based instruction (CBI) has dealt 
largely with CBI of knowledge Jislsaa, coherent bodies of essentially 
propositional knowledge. Little research has been performed on 
techniques for effective CBI of tojmis. akllla , those amalgams of 
perceptual, actor, and decision-making skills that are required by many 
real-time event-driven tasks. The demands of dynamic skill training on 
student processing resources are different from those of knowledge 
system teaching. These differences suggest that the techniques found 
to be effective in conventional CBI may not be applicable to dynamic 
skill training CBI. Two classes of research issues to be explored are 
techniques for presentation of simulation practice and methods for 
providing effective instructional feedback. 

A microcomputer-based experimental simulation training system for 
research on dynamic skill training is described. Experimental subjects 
are taught to perform a simulation task based on the Job of an air 
intercept controller. The training program permits controlled 
differences in instructional treatment for different groups of 
students, in order to explore empirical issues in dynamic skill 
training. 
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INTRODUCTION 

He intend to draw a distinction between the teaching of knowledge 
systems and the training of djmaaifc skills . By 'knowledge systems" we 
mean to refer to sets of related facts that are commonly taught as a 
body of coherent subject natter. Examples of knowledge systems include 
the history of England under the Plantagenate kings, the geological 
features of Western North America, and the multiplication tables. By 
■dynamic skills' we mean sets of intellectual processes responsible for 
selective perceptions in a real time driven context and for the 
selection and perfomance of appropriate responses in that context. 
Examples include vehicular control skills, such as driving a car or 
piloting an aircraft, and the observational and decision-making skills 
of an air traffic controller. The teaching of knowledge systems is 
characteristic of pedagogy in schools, while dynamic skills training is 
typical of many instances of on the job training. Probably as a result 
of this distribution of these two types of learning, there has been 
more research on the teaching of knowledge systems in academic research 
in education. It cannot be assumed, however, that the findings of this 
academic research will apply to dynamic skills training. 

The distinction is important because it cannot be assumed that 
those techniques which have been found to be effective for teachning 
knowledge systems will also be effective for teaching dynamic skills. 
Dynamic skill training typically involves intense student involvenent 
in the practice of the skill. Under these conditions it cannot be 
assumed that students have the same attentional resources available to 
process instructional messages. Dynamic skill training typically 
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involves practice sessions in which students must learn how to allocate 
resources to the tasks of global and local planning. Unlike typical 
knowledge system training, dynamic skill training imposes inflexible, 
strict time limitations on student responses to events. Cognitive 
processing differences in the nature of the learning processes suggest 
that effective teaching of dynamic skills requires different methods 
than effective teaching of knowledge systems. The research program 
described here is designed, in part, to test this hypothesis. 

Research on computer-based instruction and training (CBIT) has 
emphasized techniques and principles for teaching knowledge systems. 
There are a number of reasons for this emphasis. Most CBIT course 
implementations were developed to convey knowledge systems, and these 
existing courses have provided convenient test beds for research. In 
addition, many researchers are inclined by both training and experience 
to focus on the transmittal of knowledge systems such as those taught 
in academic school and university courses. Knowledge systems are not, 
however, the only class of subject matter for CBIT. A number of 
computer-based training systems are in use whose function is not to 
teach a knowledge system, but rather to convey a dynamic skill to the 
students. In particular, a significant financial committment has been 
made to dynamic skill training in the field of airplane and helicopter 
simulators for use in training pilots. These simulators can be 
expected to be made more effective as a result of basic research on 
dynamic skill training in CBIT. Yet the training of dynamic skills 
has thus far received little attention in the academic community. 

For the most part, the CBI research community has dealt only with 
the teaching of knowledge systems. Whole books have been written on 
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CAI/CBI (Burson, 1976; Taylor, 1980; and O'Neil, 1981, for example) 
without even mentioning dynamic skill training in CBIT (although others 
such as Hickey, 1975, have dealt vith the topic of simulation 
training). There is clearly a need for basic research in this 
important training area. There is scope for considerable development 
in training systems to teach dynamic skills. Vehicular control skills, 
certain production control systems in industry, and many military jobs 
(such as ground controlled landing controllers and air intercept 
controllers) require combinations of perceptual, motor, and decision 
making skills. These training opportunities can be productively 
addressed by computer based training systems, but there is reason to 
believe that the challenge cannot be met only with the pedagogical 
techniques and theories developed on the basis of research on CBI for 
knowledge systems. The attentional demands of dynamic skill simulation 
training, for example, may make conventional approaches to 
instructional feedback to student actions ineffective. 

The research community now has an opportunity to oonduct a new 
course of research on CBIT of dynamic skills. There are several 
reasons to expect this. Dynamic skill training typically implies fa^-t 
real-time pi -ceasing, often of analog inputs from the student/trainee. 
Conventional CAI implementations on large time-sharing computers, tied 
to remote terminals limited to a 300 to 1200 baud data rate over 
telephone lines, are not conducive to the development of courses on 
real-time dynamic skills. Many such dynamic tasks may require 
processor dedication to ensure prompt handling of interrupt conditions 
and other real-time aspects of the task. Recent advances in hardware 
and systems software will now make such training systems more 
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economical. On the hardware side, inexpensive microcomputer systems 
aake single-user CBIT systems financially feasible. On the software 
side, the development of a number of microcomputer operating systems 
that support faster executing and more structured high level languages 
than were formerly available make dynamic simulation programming on 
these systems both feasible and maintainable. 

Increased feasiblity of research on dynamic skill training is not 
the only reason to expect increased efforts in this area. There is a 
growing demand for practical CBIT systems in this area for training 
skills, such as vehicular or other machine system control, that require 
the fusion of perceptual, motor, and decision-making skills. In 
particular, the military services require superior training methods to 
accomplish these tasks, given the decreased level of education and 
training of entry-level personnel in the all-volunteer military. 

There is an extensive history of simulation training (or, at 
least, computer-based simulation drill and practice) in the area of 
flight training. The development of these simulation drill systems, 
frequently ^at very great expense in relation to the projected costs of 
the new generation of dynamic skill training systems, was motivated by 
compelling considerations of safety and economy. As the complexity of 
other vehicular systems (such as military tanks) increases, and as the 
cost of fuel for such systems continues to climb, it can be expected 
that there will be an increased demand for other vehicular skill 
simulators. Maximum effectiveness will be obtained from these training 
systems if they are n^t used merely as imitation environments, but also 
provide other instructional features. For example, a simulator that 
can provide tutorial interactions in addition to imitating the skill 
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environment should be able to reduce training time and Increase Its 
effectiveness by drawing the student 9 s attention to individual aspects 
of the simulation or his responses to it. 

In addition to vehicular control, other complex tasks call for the 
development of economical and effective CBIT of dynamic skills. For 
example, the Naval Training Equipment Center has contra" Ud for the 
development of experimental innovative CBIT systems to train 
Ground-Controlled Landing Specialists and Air-Intercept Controller*. 
These Jobs require that the worker not exercise direct vehicular 
control, but rather assist the pi*ot. This is accomplished through 
observations relevant to the successful control cf aircraft and by 
transmitting information and advice to the pilots of the airc;*ft. 
These real-time driven tasks require the acquisition of dynamic 
skills for successful performance. It is to be exacted that many tasks 
that call for dynamic man-machine interactions could also benefit from 
operator training .n a CBIT simulation system. The demand for such 
systems can be expected to grow when CBIT of dynamic skills adds other 
training techniques to drill and practice, and presents evidence that 
it oan make effective use of more advanced pedagogical techniques. 

Our laboratory is currently engaged in a program of research on 
the nature of dynamic skill acquisition and effective approaches to 
dynamic skill training in CBIT. Under funding from the Office of Naval 
»eaearcb<1>, a low-cost experimental data collection system for research 
on such task training has been developed. Our project has four goals: 
1* find pedagogical prlnnlnlea for CBIT of dynamic «nn^ 

There Is a substantial body of research on effective pedagogy for 
conventional CBI of knowledge systems, it is likely that many of the 
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findings of this research will also apply to instruction of dynamic 
skills. It is also likely, however, that significant departures from 
conventional approaches will be shown to be beneficial. For example* 
it is likely that classic findings on the efficacy of immediate 
feedbaok (knowledge of results) will not be upheld for highly demanding 
real-time dynamic tasks, in which the intrusion of results information 
would be expected to disrupt the training task. 

2. Modal cognitive processes in dy namic skill exercise, 

CBIT systems for dynamic skills instruction will provide an important 
source of data for psychologists interested in modeling the cognitive 
processes that underly dynamic tasks. A CBIT system based on a 
dedicated processor can record a wealth of data about a student f s or an 
expert's actions during a simulated task. This facility can be used to 
explore cognitive issues in dynamic skills, including the 
representation of skill components, using data obtained from student 
training with the experimental CBIT simulation system. 

3. Study the acquisition of complex skills. 

The acqu sition and integration of the skill components found in 
complex natu. tasks probably cannot be studied economically except 
through data acquired from a dynamic skill CBIT system. A study of the 
acquisition of vehicular control skills, for example, could be 
conducted in a much more controlled fashion through the use of a 
simulator than through observation of a student in a real vehicle. 

4. Study the consequences of voice I/O in simulation training. 
Modern dynamic simulation trainers will make increasing use of the 

new technologies for speech understanding and production. These 
features are likely to significantly modify the students 1 perceptions 
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of the responsiveness end intelligence of the computer-based systems 
with which they are interacting. For example, voice input to the 
system may contribute to an unconscious expectation on the part of the 
student that the system or some simulated component of the system will 
behave in a very human manner. If a CBIT system includes a simulated 
human fighter pilot with whom the student communicates by voice, then 
the student may be more inclined to expect the "pilot" to understand 
deviations from the presecribed voice responses than if communication 
were restricted to keyboard console interactions. The pedagogical 
consequences of such perceptions may be beneficial or detrimental, but 
they will undoubtedly require careful study. 

To work toward these goals, we have developed an inexpensive 
experimental data-collection system for research on computer-based 
training of real-time-driven tasks. The system makes use of standard, 
off-the-shelf microcomputer products, it employs the OCSD Pascal 
operating system, which provides many times the execution speed of 
interpreted BASIC and more readable and maintainable code than would 
result from the use of BASIC, Assembly Language, or Forth. The system 
includes more than 8000 lines of Pascal source code in experimental 
simulation and training programs developed in our facility. These 
programs were developed to permit the implementation of a range of 
pedagogical strategies, so that experiments with different pedagogical 
approaches can be performed. In addition, the programs have built-in 
record-keeping functions to keep track of student interactions with the 
simulation-training system. 
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TEACHING DYNAMIC SKILLS 

in gnmnle of a Dynamic Task 
In order to facilitate understanding of the dynamic task training, 
an example of a dynamio task is presented here. This is an artificial 
ta$k developed for use in our first series of experiments on dynamic 
skill training in CBIT. The task is a moderately complex simulation 
game, oalled Air Intercept Controller. It is based on the task of a 
Navy Air Interoept Controller, and includes many of the features of the 
real task. 

The student plays the role of a ground-based Air Intercept 
Controller. Figure 1 shows the student training environment. He or 
she is seated in front of two display screens, with access to a 
keyboard of special function keys and a Joystick. The screen on the 
right serves as a simulated radar display. Students observe the 
appearanoe of points of light called blips on the screen which 
represent aircraft in a defined airspace. Once every eight seconds, a 
"radar sweep" is completed and the blips are repositioned on the 
display. The second display screen and its special function keyboard 
serve as a simulated tactical data system console. The student uses 
this console, together with the associated Joystick, to acquire 
information about the speeds and headings of the aircraft represented 
by the blips on the screens. The student presses special function keys 
on the console keyboard to send directions to the student f s fighter 
aircraft. These directions include sending appropriate intercept and 
attaok headings so as to direct the fighter aircraft to correctly 
approach the enemy aircraft. When the student Judges that a fighter is 
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within firing range of an enemy craft, he or she can direct that a 
aiaale be fired at it. The student wins a game when his aircraft 
survive and and all the enemy aircraft are shot down. The task 
proceeds at a rapid pace. Even expert students are hard-pressed to 
perform all the necessary actions in order to attack and defeat a 
number of the enemy aircraft in the advanced problems. 

The experimental training system, unlike many much more elaborate 
and expensive simulator- trainers, such as those commonly used in flight 
training, is designed to do more than to provide simulated practice. It 
can also interact with the student on an instructional level. In 
addition, a detailed record of the student's interactions with the 
system, including a record of the errors made, is maintained. This 
record can serve as a basis for jzaat t&£. tutorials and replays. 

Although the experimental training system was developed as a 
research tool, it is likely that inexpensive training systems such as 
this will play a useful role in many training applications. A 
micrccomputer-based simulator might offer less superficial fidelity 
than the real environment or than a more elaborate simulator t and yet 
offer more effective training. A simulator system that includes 
facilities for tutorial and other instructional interactions combined 
with simulation may prove to be a more powerful instructional 
environment than even the most realistic simulation system that lacks 
an instructional component. Such training-simulators can be used to 
teach conceptual aspects of real-time tasks, to develop subskills 
oalled for by these tasks, or simply to teach essential safety 
procedures before high fidelity training commences. A training system 
incorporating this philosophy would make extensive use of low-fidelity 
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simulation to accomplish particular training objectives, and then 
polish the training with practice on real systems or on higher fidelity 
(and therefore uore expensive) simulator-trainers. 

Issues In the Training of Dyna mic Skills 
There are undoubtedly a large number of important issues to be 
resolved about effective approaches to CBIT of dynamic skills. Our 
research group is concentrating currently on two classes of training 
issues: presentation strategies and knovledge-of-results strategies for 
effective training. The dynamic nature of the training under 
discussion implies a wealth of presentation optics not found in 
conventional computer based instruction. The relative merit of these 
different approaches to presenting simulation training should be 
explored, A specific major training issue to explore is to isolate the 
most effective means for conveying to the student or trainee the 
quality of his or her responses to the simulation. It is expected that 
the findings of conventional CBIT for giving knowledge-of- results 
during knowledge system teaching will not apply directly to dynamic 
skill training. If they do not, then how can we give effective 
performance feedback to the students? 

Presentation Strategies 

The dynamic nature of the simulation training being considered 
here means that there iff a controllable rate at which events occur in 
the simulation. Changes in the simulated environment can take place 
more or less rapidly, or even halt, under program control, allowing the 
student to process some aspect of an ordinarily dynamic process in a 
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static Banner. The range of options dealing with changes in the pacing 
of events we call e^ant LAiS. Manipulations. A second class of 
presentation strategies are those which permit repetition of simulated 
episodes or episode segments. These we refer to as repetition 
manipulations. A third class of strategies are partial-task 
■angulations. 



Event rate m anipulations 

Event rate manipulations include stop action, slow motion, and 
fast action episode segments. The effectiveness of each of these 
techniques must be explored in a variety of simulation environments. 
For example, if slow motion simulation has a beneficial training effect 
in comparison with natural speed simulation in any context, one would 
expect to observe the favorable result in simulated contexts of rapid 
pacing that impose significant processing demands on the student. In 
addition, it is to be expected that there will be an effect due to 
locus of control (program or student initiation) of these options. 
Whether students can employ these features to their benefit in training 
is a matter for empirical determination. 



Stop action. During interactive training episodes, students 
sometimes feel that they Just need a second or two to stop and think, 
in order to assimilate something they have Just encountered or to plan 
a response. A training system that provides a student-controlled 
■stop- feature, perhaps in the form of a menu choice or a special 
"stop- key on the keyboard, would give students in dynamic simulation 
training such an opportunity to exercise some control over the pace of 
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events* Use of such en option would often mean that a student was 
using a conscious decision-making approach to the task at hand rather 
than simply reacting in an automatic manner to the information 
displayed. Whether a problem- solving orientation to the tasks of a 
dynamic skill is an effective method for acquiring that skill is an 
empirical issue. It may be that students trained with this method will 
perform less competently in the field because they become dependent on 
the use of the slower-paced problem solving method encouraged by the 
training system. On the other hand, it is possible that students would 
use th* stop action feature, and the problem-solving approach it 
encourages, only as a stepping-stone to automatic competence. If this 
is the case, the stop action feature may promote learning, particularly 
among less practiced students.- Experiments are called for to resolve 
this issue. ^ 

Stop action may also prove an effective instructional device under 
program control. During expository simulation, the task could be 
halted to direct the student's attention to some feature of the 
simulation. Instructional feedback may benefit from the use of stop 
action. Because students 1 information processing loads are likely to 
be quite high during the training of difficult tasks, the additional 
processing load imposed by instructional feedback is likely to be 
intellectually overwhelming in a continuous simulation. By 
interrupting the simulation during a feedback presentation, some 
student processing resources that were dedicated to tracking the 
simulation should become available for attending to the instructional 
feedback. 

20 
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Slow lotion and speadsd Motion. Ths participant in a real-tine 
event-driven task often senses that the action is beginning to get 
ahead of his or her ability to keep up with it. Shortly thereafter, 
the action is likely to outpace the student, and a catastrophic decline 
in performance will result. This kind of timo-driven student failure 
■ay or may not have desirable pedagogical consequences. Such failures 
■ay motivate students to automatize their processing so as to acheive 
■ore responsiveness to the task. On the other hand, these episodes may 
be confusing and discouraging for students. If tine-driven failures 
are not pedagogioally productive, an optional, student-controlled 
slow Motion node night prove useful. In the case of the Air Intercept 
Controller sinulation described above, slow notion would nean that the 
radar screen would update at longer intervals than normal, and *he 
Clips on the screen would appear to nove nore slowly than in the normal 
speed sinulation. Use of optional slow notion would lead to several 
logistic problems for the training. One would be to provide a means 

I 

for returning to normal presentation rate. If this is under student 
control, we may find that students prefer to spend too much time in 
slow notion node and actually retard their acquisition of the skills. 

I 

However, if the simulation sinply returns to normal rate after an 
arbitrarily short pe.-iod of time, the student may still be in an 
overloaded state. The best long-term solution to thii? dilema lie in 
the development of an intelligent student nonitor routine that 
estimates the student's overload state on the basi^ of the speed and 
appropriateness of his responses to the sinulation. This nonitor would 
then have eoatrol over the return to nornal speed. In fact, such a 
■onitor could be used to put the simulation into a reduced speed mode 
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when it perceived * need to reduce the student's processing load. 

In addition to use as a pacing device during practice of dynamic 
skills on simulators, slow motion may prove to be an effective 
instructional presentation tool. When a comp) icatsd new series of 
actions is to be taught to a student, he or she could be required to 
step through the sequence at slower than natural rates the first few 
times. 

Ideally, the rate of presentation of the simulation should be 
continuously variable. If a sophisticated student monitor can evaluate 
a student's momentary processing load with some accuracy, then the 
simulted rate of events could be slowed down or speeded up to match the 
student's present ability to handle the incoming data. One problem 
with this approach is that the student may actually be getting 
misinformation about the task. If one of the things a student learns 
in acquiring a dynamic skill is the natural pace of events, then an 
inconsistent simulation pace may not give the student a realistic view 
of the real world task. Only empirical study can reveal whether 
dynamic task training benefits from slow motion or fast motion 
presentations. 

partition Manipulations 

Action replays . In an action replay, a portion of a simulated 
task is repeated in order to provide a student with repeated practice 
on the elements of the skill required by that portion. One potential 
application of action replays is as a method for instructional 
feedback. When the system determines that performance on a portion of 
a practice task was not acceptable, it can require of the student that 
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that portion be repeated. This form of simulation instruction borrows 
a simple looping technique from conventional computer based 
instruction, ia which students repeat a module until they meet some 
performance criterion. 

Another possible use of action replays is under student control. 
Sometimes after completing a short sequence in a simulated event a 
student feels that he or she then understands how the sequence should 
have been done. If the student is required to continue with the next 
portion of the task immediately, the insight may be lost, and, in any 
case, there may be no immediate opportunity to apply it. If the 
student has the option to replay the sequence, applying a new 
understanding of what should be done, this extra practice could give 
the student a chance to consolidate his or her understanding. At the 
end of the replayed segment, the simulated action would continue as 
though the student had made those responses on the original segment. 

Such action replays would allow students to practice short 
sequences of actions at times when they believe the practioe will be 
useful. There may be positive motiviational effects from such replays, 
since they would allow students to avoid simulated catastrophes, if 
they observe that such a consequence is about to occur as a result of 
an error. Students could be expected to use the feature to drill on a 
sequence representative of new skill demands until they believe they 
have attained mastery. A possible negative oonaequence of the 
availability of this option is that some students may overuse the 
option, as a result of unrealistically high performance criteria. Too 
much time could be spent in training of relatively simple procedures if 
students have total oontrol over r ing through unlimited use of the 
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action replay feature. Tests of the use of this dynmaic skill training 
feature say therefore reveal a need for the system to ration use of 
this mode of training. 

Partial-task Manipulations 

In the past, computer-based simulator trainers for dynamic skills 
training, as developed for flight training and military systems 
training, have been sophisticated and very expensive devices capable of 
simulating very realistically almost the whole range of activities that 
could be performed with the real world system. Visual, auditory, and 
often proprioceptive stimuli are provided by these elaborate 
simulators. The increasing availability of inexpensive stand-alone 
microcomputer systems suggests that it may be desirable to determine to 
what extent portions of the training task can be performed by simpler 
simulation devices. 

One approach to more cost effective training might be to make use 
of trainers that are not designed to simulate the whole of the task to 
be learned, but only some portion of that task. For example, decision- 
making components of an air-intercept controller task could be taught 
separately from information-acquisition skills and from communications 
skills. The processes of information acquisition, in particular, call 
for a high degree of motor skills in this task. It is possible that 
some basic decision-making skills and resource-allocation skills could 
be taught without making use of a completely accurate representation of 
a real air intercept controller environment, using a simpler but 
functionally related simulated task. Experiments are called for to 
determine whether there is a transfer of training from a primitive 
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simulation to performance on a more sophisticated simulator or to 
performance on the real task. Part task training research has yielded 
mixed results. In some cases, the partial tasks teach response 
patterns which are antagonistic to those required in the transfer task. 
In such cases, part- task training can have a negative impact on 
performance on the target task. Part-task training is most often shown 
to be effective for cognitive, decision-making components of complex 
tasks, rather than for teaching motor responses. These findings 
suggest a course of research for part-task training of dynamic skills. 

In addition to research on traditional part-task training 
approaches, dynamic skill simulator training calls for research on 
progressive jfcasjc enlargement. In this technique, a problem or problem 
type is successively enlarged or complicated. Students are taught a 
few simple skills and are given simulated practice problems that are 
quite natural, but are unusual in that they require only the already 
presented skills. Then more skills are taught to the students and 
somewhat more complex practice problems, which call for these skills, 
are used for practice. This technique is commonly used in introductory 
computer programming classes. In this environment, students are taught 
only portions of a programming language at a time, and the problems 
they are assigned are appropriate for that portion of the language that 
they know. 

Progressive task enlargement offers a natural approach to 
incrementing a student's performance level. For dynamic skill 
training, it may provide a method for manipulating problem difficulty 
without artificially manipulating the natural rate of events as do the 
other event rate manipulation techniques discussed above. 
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Instructional Feedback 

One of the most firmly established principles of learning research 
is the importance of prompt knowledge of results following student 
responses. Feedback about the correctness of a student's interactions 
is not provided immediately by most dynamic skill simulator*. Students 
can, of course, often deduce the appropriateness of their responses on 
the basis of the simulated outcomes. If the student "pilot" of a 
simulator takes an action and shortly thereafter is told that the 
simulated aircraft has crashed and burned, then the student may 
conclude that the action was inappropriate in the circumstance. This 
type of "natural consequences" feedback has its limitations, despite 
the powerful impact it can sometimes provide. One such limitation is 
that it does not explicitly pair actions and consequences. In the case 
of the simulated crash Just mentioned, it is likely that the pilot 
would attribute the responsibility for the simulated accident to the 
last non-emergency action he took . This attribution could be 
erroneous. A natural consequences approach to feedback requires that 
students interpret their own errors. In addition, some student actions 
may be incorrect or inappropriate and yet not result in perceptible 
problems most of the time. In these cases the student will not get 
feedback about the incorrect action except under certain conditions. 

• Benefits can be expected from providing instructional capabilities 
in dynamic interactive trainee simulators, such as the air intercept 
controller discusseO above. What form such feedback augmentations 
should take is a ma;ter for basic research. Feedback techniques used 
for knowledge system teaching probably cannot be straightforwardly 
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applied to training real tine driven decision Baking tasks. One reason 
for this la that the dynamic simulation acquires its realism in part 
form the ongoing, episodic nature of the simulated events. Feedback 
(other than natural consequences) serves to interrupt the natural 
episode, and to distract the student from the simulated event. When 
knowledge system training is the goal of a computer-based simulation, 
such interruptions are no problem. Attention can be briefly diverted 
and then returned to the evidence at hand, in many cases (e.g., the 
BTL Trainer Simulator — Rigney, Towne, Moran, & Mishler, 1978; 
Munro, Towne, & King, 1980; r .jwne & Munro, 1981), the student can 
replay portions of the simulation in order to reconstruct his findings 
if his memory is disrupted. Such an approach is not ordinarily 
possible in the simulation of a dynamic event. 

The experimental system described below has been developed in part 
to explore the pedagogical consequences of two aspects of the 
presentation of feedback information in dynamic skill training. 
The first aspect is the continuity of the simulation during feedback 
episodes. The second aspect of feedback presentation to be studied is 
student control over MtiSH feedback will be presented, a factor we will 
refer to as feedback intruaivenasa . 

In order to anticipate the impact of either simulation continuity 
or feedback intrusiveness upon training effectiveness we need to 
consider the demands which are placed upon the cognitive resources of 
the student when confronted with a dynamic skills training task. Two 
primary types of cognitive skill are likely to be affected by these 
training task variables. First, during a dynamic skills training task 
the student's attentional resources are often heavily taxed. Students 
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■ust attend to the flow of events in the simulation, and they must 
attend to other essential training inforcst*on, such as feedback* 
Second, students 9 enooding and representational skills are exercised to 
produce accurate cognitive representations of the task and Its 
structure, in order to produce immediately required responses and to 
plan for future contingencies. The factors of simulation continuity 
and feedback intrusiveness must be evaluated in terms of their impact 
upon attentional and representational skills. 

Simulation Continuity 

Consider the following two strategies for simulation continuity 
during administration of instructional feedback; the training simulation 
can be stopped until the feedback is complete or it can be allowed to 
continue normally during presentation of the instructional feedback. 
Consideration of the cognitive factors just mentioned — attentional 
monitoring and representation formation — seem to lead to contrary 
points of view about the relative effectiveness of these two strategies 
for presenting the simulation. 

With respect to the attentional processes of the student, the 
interrupted simulation strategy has the potentially beneficial feature 
of reducing the attentional load during feedback episodes by 
eliminating the need to attend to the simulation when feedback 
information is being presented. In the case of a continuously 
presented simulation, a student's processing capacity is likely to be 
largely taken up with attending and responding to the simulation. The 
addition of the task of attending to the instructional message while 
the simulation continues is likely to overload attentional mechanisms, 
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to the detriment of both performanoe on the simulation task and 
attention to the feedback. It is therefore reasonable to expect that 
performance under training conditions in which the simulation is 
interrupted during feedback episodes will be superior to performance 
when presentation of the simulation is continuous throughout the 
training session. 

On the other hand, the formation of a precise and accurate 
representation for a task in which there are real time elements may be 
hindered by frequent interruptions of task continuity. It is reasonable 
to expect that a complete mental representation of a task involving 
coordination of responses to real time events must contain some accurate 
portrayal of the temporal relationships among the various task elements. 
That is, if one of the things students learn is the rate at whioh a 
series of simulated events is to occur, then the interruption of this 
pacing may be^iaruptive to the representational mechanisms. There is no 
direct experimental evidence for such representational mechanisms, at 
least in the context of training. However, the studies by Shepard and 
his colleagues (Shepard & Metzler, 1971; Cooper & Shepard, 1978) suggest 
that mental /Imaginal processes can provide detailed representations for 
certain types of continuously variable physical events such as rotation 
of a physical object. It is an open question whether such representational 
9 y llities extend to the diversity of temporal relationships which exist 
in a typical dynamic skills task. In addition, it is possible that 
a student could overcome the disruptive effects of interrupting the 
simulation ty performing some kind of "mental subtraction" of any feedback 
time which intervened between two simulation events to be temporally 
related. Therefore, it is possible, but by no means certain, that 
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interrupting the simulation during feedback episodes will intefere with 

the student 1 a ability to develop a complete and accurate representation • 

of the simulated task. 
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Feedback Intrusiveness 

The second i ^search issue in instructional feedback for dynamic 
skills training is feedback intrusiveness and its effect upon a 
students attentional and representational cognitive processes. 
Feedback Intrusivenes s refers to whether the student controls the time 
at whici; feedback information is presented in a training task. 
Feedback under student control is termed non-intrusive feedback ,, and 
externally paced presentation of feedback is termed intrusive . This 
terminology is motivated by noting an analogy of student-training system 
interaction to a conversational context. 

The students interaction with the dynamic skills CBIT system has 
parallels to a conversational interaction between two people. In 
normal conversation, the participants exchange indications of their 
readiness to accept input from each other; that is, they signal 
turn-taking in the conversation. In conventional knowledge system 
teaching in CBI, each student response (such as typing in the answer to 
a question) can be thought of as a signal that the student has 
surrendered a conversational turn. After making a response, the 
studtnt expects a reply from the teaching system. In dynamic skill 
training, however, each student interaction with the simulation system 
is not a conversational turn surrender. Instead, the student remains 
actively engaged with the simulation, preparing a series of actions in 
response to observed and expected simulated events. In this context, 
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the interjection of an instructional feedback message is an intrusion. 
The system is, in some sense, breaking in on the student's turn. 

Consider the impact upon attentlonal processing of providing 
feedback by intrusive versus non-intrusive means. During a complex 
training session, a student v s processing resources are likely to be 
largely absorbed in attending and responding to the task simulation 
itself. If the task is suddenly intruded upon — whether or not the 
simulation is stopped during the feedback episode ~ the intrusion is 
likely to require additional processing resources to perform the 
attentlonal shift. This surge in processing resource demand is likely 
to interfere with the normal learning and performance processes. If 
the disruption occurs at a point in the task when a large percentage of 
oognitive resources are already committed, then either or both of the 
attention to the feedback and task performance are likely to suffer. 
The student will presumably shift attention from the task simulation to 
feedback information only at points where processing requirements are 
relatively low so that the attentlonal shift may be accomplished with a 
minimum of disruption to response and learning processes. If the 
argument Just given Is sound then we expect that feedback given 
intrusively will be more disruptive of both learning and performance 
than will feedback provided non-intrusively. 

These points about attentlonal processes underscore our analogy 
between conversation and student interaction with the training system. 
It seems likely that conversational oooperatlon is required for reasons 
quite like those Just discussed for skills training. In this view, 
conversational cooperation is an adaptation to information processing 
resource limitations. Therefore, it is our expectation that a dynamic 
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skills training system that partially emulates some of the features of 
conversational turn-taking in instructional feedback will prove 
superior to a system that arbitrarily interrupts student task 
processing with instructional messages. This emulation consists of 
signaling a readiness to provide feedback in a non-intrusive way, and 
then postponing the presentation of the message until the student 
explicitly surrenders a turn and requests presentation of feedback. 

It is plausible to claim that the non-intrusive feedback condition 
(student paced presentation of feedback) could present difficulties for 
representational (as opposed to attentional) processes. This is due to 
the fact that the student will frequently wait to see a feedback 
message until some time after the context to which the message refers 
has passed. The traditional results on delay of reinforcement during 
learning indicate such a delay will be detrimental to learning. Thus, 
if feedback serves as a reinforcer (positive or negative) the delays 
which occur in the non-intrusive conditions should impair overall 
student performance. We can put this in a more cognitive perspective 
by noting that when the feedback message is delayed it should be 
more likely that the content of such a message will not be correctly 
related to the student's representation of the task. The student could 
readily be confused as to the context to which the feedback message 
refers. Thus, we have two types of argument that students in the 
non-intrusive condit* n will be at a disadvantage in making use of the 
feedback information. The students whose feedback is presented 
intrusively always receive the feedback message immediately following 
the oontext for whish it is generated and 30 should not suffer this 
representational and reinforcement disadvantage. 
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These arguments ere challenged i' we postulate that students have 
some ability to reinstate the context to which a feedback message 
refers froa the oontent of that aessage. The research specifically 
addressed to delay of knowledge of results tends to support this point 
of view (Kling * Schrier, 1971). In these studies it appeared that 
making students wait for feedback was not detrimental if little or no 
responding wis required during the wait interval, a cognitive 
interpretation of these findings is that students were (due to memorial 
processes) able to maintain a trace of the context for the knowledge of 
results information. If students in a dynamic skills training task 
were similarly able to recall the context of a feedback message, then 
it would follow that students receiving feedback non-intrusively would 
not suffer a difficulty. Therefore, it is an open question as to how 
the variable of intrusiveness will affect students in a dynamic skills 
task. Attentional demand considerations suggest superior learning 
performance for students in a non-intrusive feedback condition. 
Representational and reinforcement considerations argue for the 
conclusion that students' performance in the non-intrusive feedback 
condition will be adversely affected unless the feedback messages they 
receive are sufficiently clear that they reinsatate the referent 
context. 
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A JBttiKIL XflL r— dhmok RflMlTQfa. 

The table below represents a two factor deaign for studying the 
•ffacts of simulation continuity during feedback and feedback 
intrusiveness. 



Continuity 



Interrupted 




1 I 


2 


Continuous, 


Visible 1 


3 I 


4 


Continuous, 


Invisible I 


5 I 


6 



Intrusive 



Non-intrusive 



Intruaiveneaa 

$ix different conditions for the presentation of instructional feedback 
■essa ys are presented here: 

1. Intrusive feedback with interrupted simulation. The student has no 
control over presentation of the feedback messages. When the system 
detects a student error or a condition that demands student attention, 
it freezes the simulation, sounds an audible feedback warning tone and 
presents the feedback message in a reserved area of a display screen. 
The simulation remains frozen until the user "accepts" the feedback by 
depressing a special key. At that point the simulation resumes. 

2. Hon-intrusive feedback with interrupted simulation. When the 
training system detects a student error or other condition that 
warrants instructional feedback, it sounds an audible feedback warning 
tone. The simulation continues without interruption. When the student 
feels ready to see the feedback message, a special "help" key is 
depressed. At this point the simulation freezes and the student can 
read the message. The user then "accepts" the feedback by depressing 
another special key. At that point the simulation is unfrozen and 
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oontinues normally. 

3. Intrusive feedback with continuous simulation, in this mode of 
instruction, the system presents feedback messages, as in 1, whenever 
their conditions occur. The simulation is not interrupted, however, 
during presentation of the instructional message. 

4. Non-intrusive Redback with continuous simulation. The system 
signals the availability of feedback messages, but presents them only 
upon student request, as in 2. As in 3, the simulation is not 
interrupted during feedback presentation. 

5. Intrusive feedback with continuous but invisible simulation. As in 

1 and 3, the instructional messages are presented whenever their 
conditions arise. The simulation screen goes blank for the student 
during the feedbacK. When feedback ends, the simulation becomes 
visible again, but in the state to which it advanced during the 
feedback presentation. 

6. Non-intrusive feedback with continuous invisible simulaton. As in 

2 and 4, instructional messages are presented at student request. The 
simulation continues in an invisible mode until the student accepts the 
feedback message. 

The first goal of our research project is to explore the 
consequences of different combinations of the simulation continuity and 
feedbaok intrusiveness factors on the success of dynamic skill 
training. To that end, an experimental test bed training system has 
been developed. The features of this system are described in the next 
section. 
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THE EXPERIMENTAL SYSTEM 
Figure 2 presents the functional structure of our experimental 
dynamic skill training system. A student training episode is viewed as 
having two major components. The first is a pre-training session, in 
which the student is introduced to the task and to the activities that 
comprise the skill. The second is a practice training session, in 
which the student receives a combination of simulated practice and 
tutorial. These tutorial components are not separate from the practice 
episodes, but are intertwined with the practice activities. One of the 
research questions being pursued with the system is how to most 
effectively mix practice and tutorial instruction. 

A complex set of data is recorded during each student practice 
training session. This data set includes records of student actions, 
of the occurence of certain simulated events, of detected student 
errors, and of tutorial interactions with the student. A set of data 
extraction programs are applied to these data to produce data relevant 
to research issues of interest. These extracted data are then analyzed 
with the SPSS data analysis package on the University mainframe 
computer. 

The Simulation Trainer £cfigUBL 

The current version of the dynamic skills trainer program 
simulates a game-like task similar to that of an air intercept 
controller. Students are trained on the task in a single training 
session of approximately four hours, training consists of an 
introductory session of one and one half hours in which the student 
sees viedeotaped explanations of the task and the equipment and works 
through an introductory training program. This introductory program 
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requires that the student read brief explanations of each of the 
eontrols available to the student on the simulated eontrol oonsole. 
After reading eaoh explanation, tb* student is required to briefly 
interaot with a siaple simulation that requires the exeroise of the 
Just-discussed function. Students who fail to correctly use any of the 
controls are required to repeat the nodule that explains how the 
oontrol is used. 

After viewing the introductory videotaped sequences and completing 
the introductory training program, students practice the skills by 
interacting with 1 main simulation training program. The student 
spends about two and one half hours in a series of thirty training 
exercises in this program. These exercises are blocked (5-10-10-5) in 
blocks of increasing difficulty, with problems of approximately equal 
difficulty within a block. Appendix II gives a detailed narrative 
description of a simple problem interaction. 

The simulation program has the top level structure depicted in 
Figure 3. Note that the system uses a system of frequent checkn of the 
real time clock for pacing the simulation rather than clock-based 
interrupts. This feature facilitates straightforward structured 
programming in Apple Pascal. The heart of the simulation is the 
Probloop procedure, which constantly ohecks for student errors, for new 
student actions, and for new simulated events. The form of the 
exeroises are determined by coded data files that prescribe the 
simulated events that are to occur in each exercise. An exercise comes 
to an end when either the student or the simulated- enemy aircraft win 
or the time alloted for the exercise in its coded file is expended. 
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SUMMARY 

We have shown that dynamic skill training differs in important 
ways from knowledge system instruction. The demands placed upon 
students' cognitive processing resources are different in these two 
types of learning. These differences suggest that computer based 
training of dynamic skills offers scope for research projects that will 
differ in methods and results from conventional computer based 
instruction research. Issues for dynamic skill training in particular 
need of empirical resolution include finding the most effective 
simulation presentation techniques and the most effective means for 
providing instructional feedback. Dimensions of presentation dynamics 
to be explored include manipulations of the rate of simulated events, 
repetition techniques, and partial-task training approaches. 
Instructional feedback issues include whether simulation should 
continue during feedback and whether students or the training programs 
should have control over the presentation of feedback. 

In order to address these and other issues in dynamic skill 
training t a low cost experimental simulation training system has been 
developed. This system trains experimental students to perform a task 
similar in many respects to that of an air intercept controller. The 
training program permits controlled differences in instructional 
treatments for different subjects. The system is being used as a tool 
for the empirical resolution of dynamic skill training issues. 
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Figure 2. The functional structure of the experimental dynamic skill 
training system 
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APPENDIX 1 
HlTdWir* UUl ODT«tln? Svatem 
Figure 1 diagrams the hardware of the experimental systeo for 
conducting reaearch on computer-based training of real-time driven 
tasks. Only the major, system-level components are shown here. 
The shaded elements are planned for but not currently integrated into 
the complete hardware/software package. An Apple II micro-computer is 
the heart of the system. This computer uses the Apple Pascal operating 
system, within which the simulation- training programs have been 
written. A Corvus 10-megabyte Winchester technology hard disc drive 
provides mass storage for the system. The speed of this drive in 
accessing and transferring data is very important to the success of the 
simulation. The program uses overlaid segments, so fast disc transfers 
are important^ maintaining a realistic pace in the simulation. The 
Corvus disc drive provides a 16K RAM buffer which typically holds the 
next required program segment. As a result, overlays from the iisc 
drive are accomplished very quickly. 

Three display screens are used for the simulator trainer. The 
first of these is a standard RS-232C-compatible 80 column CRT terminal. 
The keyboard of the terminal has been modified with special key caps 
corresponding to simulated functions. This display serves as the 
alphanumeric console of a simulated radar-control computer. The second 
display screen is a high bandwidth green phosphor video monitor. This 
screen displays the medium-resolution graphics output of the Apple II. 
It functions as a simulated radar screen in the CBIT system. The third 
screen is a color monitor used to display videotaped instructional 
sequences under, computer control. The computer-based videotape 
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controlling device (t Cavri V controller), which permits random-access 
of videotaped segments, also has the capability of controlling a 
videodisc player. 

Student interaction with the simulator trainer is by Mans of key 
presses of the specially ooded key caps on the CUT terminal and through 
the use of the linear Joystick connected to the Apple II. In addition, 
the system will sake use of voice input and output technology in future 
experiments. An internal real time clock permits acourate recording of 
the times of student actions 
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APPENDIX II 
The Task 

A narrative description can only partially convey the nature of 
the simulation, but one is attempted here. A problem of relatively low 
difficulty might consist of the following events. The Radar screen is 
initially blank. A few seconds later, the first radar sweep takes 
place, and a blip appears in the lower left corner of the radar screen. 
The student depresses a key labeled "JOY" in order to activate his 
joystick. When the joystick is activated, a small crosshairs cursor 
appears on the radar screen. The student manipulates the joystick to 
position the cursor on the blip. A new radar sweep occurs (eight 
seconds after the previous one) and the student must reposition the 
cursor, using the joystick. With the cursor centered, the student 
•hooks" the blip by depressing a symbol key, "C1", on the console 
keyboard. The simulated IFF function (Identification: Friend or Foe) 
determines that the symbol is appropriate for the blip, so the symbol 
"C1» ^appears on the radar screen next to the blip. The Joystick cursor 
disappears, because pressing the symbol key (C1), turned off the 
joystick. Another radar sweep takes place, and the blip advances again 
toward the upper right corner (NE). The symool has been left behind 
because the simulated tracking computer requires two fixes to project a 
course for the symbol. The student reactivates the joystick by 
depressing the "JOT" key, and goes through the same process of hooking 
the blip with its symbol again. Now the system draws a track for the 
blip and advances the symbol on the projected path with each radar 
update. (Vote that if the simulated pilot of the blip ohanges the 
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course or speed, the symbol and the track will continue on the old 
course until the student rehooks.) 

Once the student has hooked and rehooked the blip, he or she will 
depress the "FUL" and "WPN" keys to find out aircraft CVs fuel and 
weapons status. These figures will be posted in the appropriate 
display area of the simulated tracking computer console display screen. 
The student then observes a second blip appearing near the top of the 
screen. The Joystick is activated and the student attempts to hook the 
new blip with the "C2" symbol. A beep from the console indicates that 
the simulated IFF function has determined that C2 is not an appropriate 
designation. The student now depresses the "B1" symbol key, and the 
symbol appears. This means that the new blip is a Bogey (a bad guy). 
The student reactivates the joystick, and, after the next screen 
update, rehooks the B1 blip. Now that the tracking computer has a fix 
on the location and course of the Bogey , an intercept can be computed. 
The student depresses the gray "INT" key. On a reserved portion of the 
iimulated tracking computer console display the student sees a request 
for the symbol of the combat air patrol to be assigned to the 
intercept. He depresses C1. The system then asks for the Bogey to be 
intercepted. The student enters B1. The system then displays a 
recommended intercept heading, say 270 degrees. The student must no* 
send this recommendation to the pilot of CI quickly. This is done by 
depressing the yellow "INT" key and following the system* s directions. 

As the two blips near each other, the Bogey may take evasive 
action, which the student must respond to by making the simulated 
tracking computer recompute intercept or attack headings. Attack 
headings must be computed when a fighter is within attack range of a 
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Bogey. The new recommendations oust, of course y be relayed to the 
pilot. Finally, when the student Judges that the fighter is within 
firing range of the blip, he directs it to fire a missle. If the 
fighter is within range and on the correct attack course, the Bogey is 
downed and the student wins the exercise. The student can lose an 
exercise either by allowing the Bogey to shoot down the combat *ir 
patrol or by allowing the Bogey to escape. 

The exercise dscribed was a simple problem in that only one Bogey 
and one combat air patrol were present. The example was further 
simplified in that events were regularly paced and the student did not 
make mistakes that might have resulted in the presentation of 
instructional feedback messages. 
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